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Abstract — This paper is concerned with the problem of 
Model Predictive Control and Rolling Horizon Control of 
discrete-time systems subject to possibly unbounded random 
noise inputs, while satisfying hard bounds on the control inputs. 
We use a nonlinear feedback policy with respect to noise 
measurements and show that the resulting mathematical pro- 
gram has a tractable convex solution in both cases. Moreover, 
under the assumption that the zero-input and zero-noise system 
is asymptotically stable, we show that the variance of the 
state, under the resulting Model Predictive Control and RolUng 
Horizon Control policies, is bounded. Finally, we provide some 
numerical examples on how certain matrices in the underlying 
mathematical program can be calculated off-line. 

I. Introduction 

Model Predictive Control (MPC) for deterministic systems 
has received a considerable amount of attention over the 
last few decades, and significant advancements have been 
realized in terms of theoretical analysis as well as industrial 
applications. The motivation for such research thrust comes 
primarily from tractability of calculating optimal control laws 
for constrained systems. In contrast, the counterpart of this 
development for stochastic systems is still in its infancy. 

The deterministic setting is dominated by worst-case anal- 
ysis relying on robust control methods. The central idea is 
to synthesize a controller based on the bounds of the noise 
such that a certain target set becomes invariant with respect 
to the closed-loop dynamics. However, such an approach 
usually leads to rather conservative controllers and to large 
infeasibility regions, and although disturbances are not likely 
to be unbounded in practice, assigning an a priori bound to 
them seems to demand considerable insight. A stochastic 
model of the disturbance is a natural alternative approach 
to this problem: the conservatism of the worst-case analysis 
may be circumvented, and one need not impose any a priori 
bounds on the maximum magnitude of the noise. However, 
since in practice control inputs are almost always boimded, 
it is of great importance to consider hard bounds on the 
control inputs as essential ingredients of the controller syn- 
thesis; probabilistic constraints on the controllers naturally 
raise difficult questions on what actions to take when such 
constraints are violated (see however [IJ for one possible 
approach to answer these questions). 

In this paper we aim to provide answers to the following 
questions: Given a linear system that is affected by (possibly 
imboimded) stochastic noise, to be controlled by applying 

This research was partially supported by the Swiss National Science 
Foundation under grant 200021-122072 

The authors are with the Automatic Control Labo- 
ratory, Electrical Engineering, ETH Zurich, Switzerland 
hokayem, chatterjee, lygeros@control . ee . ethz . ch 



predictive-type bounded control inputs, (i) is the associated 
optimization problem tractable? (ii) under what conditions 
is stability (in a suitable stochastic sense) of the closed-loop 
system guaranteed? (iii) is stability retained both in the case 
of MPC implementation and the case of RolUng Horizon 
Control (RHC) implementation? 

In the deterministic setting, there exists a plethora of 
literature that settles tractability and stability of model-based 
predictive control, see, for example, [2], [3], [4], [5] and 
the references therein. However, there are fewer results in 
the stochastic case, some of which we outline next. In [6], 
the authors reformulate the stochastic programming problem 
as a deterministic one with bounded noise and solve a 
robust optimization problem over a finite horizon, followed 
by estimating the performance when the noise can take 
unbounded values, i.e., when the noise is unbounded, but 
takes high values with low probability (as in the Gaussian 
case). In [7], [8] a slightly different problem is addressed in 
which the noise enters in a multiplicative manner into the 
system, and hard constraints on the state and control input 
are relaxed to probabiUstic ones. Similar relaxations of hard 
constraints to soft probabilistic ones have also appeared in [9] 
for both multiplicative and additive noise inputs, as well as 
in [10]. There are also other approaches, for example those 
employing randomized algorithms as in [11], [12]. Finally, 
a related line of research can be found in [13], and a novel 
convex analysis dealing with chance and integrated chance 
constraints can be found in [14]. 

In this paper we restrict attention to linear time-invariant 
controlled systems with affine stochastic disturbance inputs. 
Our approach has three main features. Firstly, for the finite- 
horizon optimal control subproblem we adopt a feedback 
control strategy that is affine in certain bounded nonlinear 
functions of the past noise inputs. Secondly, instead of 
following the usual trend of adding element-wise constraints 
to the control input in the optimization, we propose a new ap- 
proach that entails saturating the utilized noise measurements 
first and then optimizing over the feedback gains, ensuring 
that the hard constraints on the input will be satisfied by 
construction. This novel approach does not require artificially 
relaxing the hard constraints on the control input to soft 
probabilistic ones to ensure large feasible sets, and still 
provides a solution to the problem for a wide class of 
noise input distributions. In fact, we demonstrate that our 
strategy (without state constraints) leads to global feasibility. 
The effect of the noise appears in the finite-horizon optimal 
control problem as certain covariance matrices, and these 
matrices may be computed off-line and stored. Thirdly, the 
measurement saturation functions are only required to be 



elementwise bounded in order to ensure tractability of the 
optimization problem while maintaining hard constraints on 
the control input; therefore, these measurement saturation 
functions may be picked from among the wide class of 
saturation functions, the standard sigmoidal functions and 
their piecewise affine approximations, etc. 

Once tractability of the finite-horizon underlying optimiza- 
tion problem is insured, it is possible to implement the 
resulting optimal solution using an MPC approach or an RHC 
approach. In the former case [2], the optimization problem 
is resolved at each step and only the first control input is im- 
plemented. In the latter case [15], the optimization problem 
is resolved every N steps (with being the horizon length) 
and the entire sequence of N input vectors is implemented. 
Both of these approaches are shown to provide stability under 
the assumption that the zero-input and zero-noise system is 
asymptotically stable, which translates into the condition that 
the state matrix A is Schur stable. At a first glance, this 
assumption might seem restrictive. However, the problem of 
ensuring bounded variance of linear Gaussian systems with 
bounded control inputs is, to our knowledge, still open, and 
here we are considering the problem of controlling a linear 
system with bounded control input and possibly unbounded 
noise. It is known that for discrete-time systems without any 
noise acting on the system it is possible to achieve global 
stability if and only if the matrix A is neutrally stable [16]. 

This paper unfolds as follows. In f|ll] we state the main 
problem to be tackled with the underlying assumptions. In 



^III we provide a tractable approach to the finite horizon 
optimization problem with hard constraints on the control 



input, as well as some examples in [III-A Stability of the 



MPC and RHC implementations is shown in \ IV and hints 



onto the input-to-state stable properties of this result are 



provided in \ IV-C Finally, we provide a numerical example 
in S|V]and conclude in §VI| 

Notation 

Hereafter, N := {1,2,...} is the set of natural numbers. 
No := N U {0}, and Mjjq is the set of nonnegative real 
numbers. We let Ia(') denote the indicator function of a set 
A, and I„xn ™d Onxn denote the n-dimensional identity 
and zeros matrices, respectively. Also, let E^.^ [•] denote the 
expected value given xq, and tr(-) denote the trace of a 
matrix. For a given symmetric n-dimensional matrix M with 
real entries, let {Ai(M) | i — l,...,n} be the set of 
eigenvalues of M, and let A„iax(M) :— maxiAi(M) and 
\mn{M) :— mini \ {M). Let ||-||p denote standard norm. 
Finally, the mean and covariance matrix of any vector v are 
denoted by S^, and respectively. 

II. Problem Statement 

Consider the following general affine discrete-time 
stochastic dynamical model: 

xt+i^ Axt + But + Fwt + r, i G No, (1) 

where xt G M" is the state, ut G M™ is the control input. 



known matrices, and r e M" is a known constant vector 
We assume that the initial condition xq is given and that, at 
any time t, xt is observed exactly. We shall assume further 
that the noise vectors wt are i.i.d. and that the control input 
vector is bounded at each instant of time t, i.e., 

WteU:={t.eM™|||u|U<C/,„ax} VteNo, (2) 

where C/,„ax > is some given element-wise saturation 
bound. Note that the model ([T]i with constraints (|2]i can 
handle a wide range of convex poly topic constraints. In 
particular, any system 

xt+i = Axt + Bvt + Fwt + f (3) 

with input constraints Vt that can be transformed to the 
form Q by an affine transformation 

Vt = Suf + I 

is amenable to our approach by setting B — BS and r = 
Bl + f in ([T]). Note that the set V need not necessarily be a 
hypercube, or even contain the origin. Note also that we can 
assume that wt is zero mean in ([TJ without loss of generality; 
given a system of the form (|3]l where wt is not zero mean, 
we can replace it by a system in the form ([TJ with zero mean 
in which 

Wt ^ Wt ~ E[wt] 

by setting r = f + FE[wt]- 

Fix a horizon N G N and set < = 0. The MPC procedure 
can be described as follows. 

(a) Determine an admissible optimal feedback control pol- 
icy, say 7r^.j_|_jv_]^ G 11, for an A^-stage cost function 
starting from time t, given the (measured) initial con- 
dition Xt; 

(b) increase t to t + 1, and go back to step (a). 

On the other hand, the RHC procedure simply replaces (b) 
above by 

(b') apply the entire sequence Tr^.f_^j^_-^ of control inputs, 
update the state xt+N at the {t+N~l)-th step, increase 
t to t + N and go back to step (a). 
Accordingly, the t-th step of this procedure consists of 
minimizing the stopped iV-period cost function starting at 
time t, namely, the objective is to find a feedback control 
policy that attains 



inf Vt,t+N^ 



i{Tr,x) 



ujRiUi 



^t+N^t+N^t+N 



(4) 



Since both the system ([TJ and cost ([4]) are time-invariant, it 
is enough to consider the problem of minimizing the cost 
for i = 0, i.e., the problem of minimizing Vo_Ar-i(7r, x) over 

TT e n. 

In view of the above we consider the problem 

(xjQtXt + ujRtUt) + xJfQNXN 



min Ex 
Tren 



' (5) 



Wt € M" is a stochastic noise input vector, A, B and F are s.t. dynamics ([TJ , and constraints ([2j 



where Qt > and Rf > are some given symmetric 
matrices of appropriate dimension. If feasible with respect to 
(|2]i. Problem (|5]) generates an optimal sequence of feedback 
control laws tt* — {u'q, ■ ■ ■ , u*J^_-^^}. 

The evolution of the system ([T]i over a single optimization 
horizon N can be described in compact form as follows: 



= Axn + Bu + DFw + Df, 



(6) 



where 
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InXn, 



F := 



F 



where the input 

MeU:={eGffi^"||lelU<C/max}. 



(7) 



Using the compact notation above, the optimization Prob- 
lem (|5]l can be rewritten as follows: 

min Ej. Ix'^^Qx + Ru] , 
s.t. dynamics ([6| , and constraints ([7| , 



where 



Qo 



Onxn 



Q 



N 



R = 



Rn 



Rn-1 



The solution to Problem ([Sj is difficult to obtain in general. 
In order to obtain an optimal solution to Problem ([SJ over 
the class of feedback policies, we need to solve the Dynamic 
Programming equations. This generally requires using some 
gridding technique, making the problem extremely difficult 
to solve computationally. Another approach is to restrict 
attention to a specific class of state feedback policies. This 
will result in a suboptimal solution to our problem, but may 
yield a tractable optimization problem. It is the track we 
pursue in the next section. 



III. Tractable Solution under Bounded Control 
Inputs 

By the hypothesis that the state is observed without error, 
one may reconstruct the noise sequence from the sequence 
of observed states and inputs by the formula 



Fwt 



xt+i - Axt - But 



t £ No. 



(9) 



In the light of this, and inspired by the works [17], [18], we 
shall consider feedback policies of the form: 



Uf 



J2 Gt.^Fw, + dt 



(10) 



4=0 



where the feedback gains Gt.i E M™^" and the affine terms 
dt € M™ must be chosen based on the control objective, 
while observing the constraints (|2|. With this definition, the 
value of M at time t depends on the values of w up to time 
t—1. Using (|9]l we see that Ut is a function of the observed 
states up to time t. It was shown in [18] that there exists a 
one-to-one (nonlinear) mapping between control policies in 



the form ( 10 1 and the class of affine state feedback policies. 
That is, provided one is interested in affine state feedback 
policies, parametrization (|9]l constitutes no loss of generality. 
Of course, this choice is generally suboptimal, but it will 
ensure the tractability of a large class of optimal control 
problems. In compact notation, the control sequence up to 
time — 1 is given by 



= GFi 



d, 



(11) 



where d := [dj 

Or 



dj 



djf_j] , and 



G 



Gil 



Gn-1,0 



Gn-i,n-i 



Or 



Since the elements of the noise vector w are not assumed 
to be bounded, there can be no guarantee that the control 
input ( [TT| will meet the constraint (|7]). This is a problem 
in practical applications, and has traditionally been circum- 
vented by assuming that the noise input lies within a compact 
set [18], and designing a worst-case controller. In this article 
we propose to use the controller 



instead of ( |TT] i, where 

^{Fw) = 



G(p{Fw) + d, 
(Pq{Fwo) 

(fN-liFwN-l)^ 

ipi{Fwi) is a shorthand for 
[ip}{Fiw,),...,^f{F^w^)]'^, Fj is the 
the matrix F, and cpj : M. 

sup \ipl{s)\ < 0„iax < Un- 



(12) 



the vector 
j-th row of 
M is any function with 
In other words, we have 



chosen to saturate the measurements that we obtain from 
the noise input vector before inserting them into our 



control vector. This way we do not assume that the noise 
distribution is defined over a compact domain, which is an 
advantage over other approaches [6], [18]. Moreover, the 
choice of element-wise saturation functions (pi{-) is left 
open. As such, we can accommodate standard saturation, 
piecewise linear, and sigmoidal functions, to name a few. 

Remark 1: Our choice of saturating the measurement 
from the noise vectors renders the optimization problem 
tractable as opposed to just calculating the whole input 
vector u and then saturating it afterwards, which tends to 
an intractable optimization problem. < 

Remark 2: Note that the choices of control inputs in ^H} 
and ( [T2] i are both non Markovian; however, they differ in 
the fact that the former depends affinely on previous noise 
inputs w, whereas the latter is a nonlinear feedback due to 
passing noise measurements through the function (p{.). < 

Proposition 3: Assume that Exo 0, Vxo € 

R". Then, Problem ([8| with the input ([% is a convex 
optimization problem, with respect to the decision variables 
(G, d), which is given by 



b^d + d^Mid + tr (G'^MiGAi + M2GA2 

s.t. M.| + ||G,|L 



min 

(G,d) 



Nm 
(13) 



where G, is the i-th row of G, 



= 2(Ato + DF^i^ + ffQE, Ml = i? + B'^QB, 
M2 = 2F'^D^QB, 

Ai = dia.g{E[tpo{Fwo)(po{Fway] , • • • , 

E[(pN-l{FwN~l)^N^l{FwN~l)~''] }, 

A2 = diag{E[(^o(^'wo)u;o], • • ' , 

Proof: Let us first consider the cost function in Problem 
|8] After substituting the system equations, we obtain 

Exg[x'^Qx + iJRu] = (14) 

Ea; J(Aa;o + Bu + DFw + f)^ Q {Axo + Bu + DFw + r) 

= {Axo + r)'^Q{Axo + f)+ 2{Axo + fYODFE^^ [w] 
+ 2{Axo + ffQBE,^ [u] + 2E^, [w^ F'^ D'^ QBu] 
+ E^„ [w^F'^D^QDFw] + E^ro [v^{R + B^QB)u\ . 

Note that since E^.^ [(^(f'u))] = 0, we have that E^^^ [u] = d. 
Accordingly, using the definitions of b. Mi, M2, and A2, 

E^„ [x^Qx + v^Ru] = b~^d + tr(M2GA2) + c 

+ E.,,[u'Miu], (15) 

where c= {Ax:Q_+_fy Q{Axa + f) +tv{F'^ QOFY^ta) + 
2{Axi^ + QDF flu, is a constant that we omit as it does 
not change the optimization problem, and we have used the 
following intermediate step 

E,„ [w'F^D'^QBu] = E,„ [w'f'^D'^QB{G^{Fw) + d)] 
= tr{F'^ D'^QBG A2) + ^il,F'^ QBd. 



Using again the assumption that E^-q [<^{Fi 
that 



0, we have 



E,„ [u'Nhu] = E,„ [{G^{Fw) + dYMi{G^{Fw) + d)] 
= E^„ [(p{Fw)^ G'^ MiG^{Fw)\ + d^ Mid 
= tr(G"^MiGE^„ [(p{Fw)(p{FwY] ) + d^ Mid 
= tr(G"^MiGAi) + (FMi J. (16) 

Finally, combiiung ( [T5| ) and ( fTS] ), we obtain the cost in 
Problem [13] which is convex. 

Let us look at the constraints in Problem [8] The proposed 
control input ( [T2] i satisfies the hard constraints (|7]i as long 
as the following condition is satisfied: ||J + G(^('i()) ||^ < 
C^max, V(^(u;) such that \\^{w)\\^ < 0max- TWs is equiv- 
alent to the following conditions: Vi — 1, • • • , Nm, \di + 

Gi<^(w)| < [/max, V(^(w) such that < (/)max- 

As these conditions should hold for any permissible value 
of the function (fiw), we can eliminate the dependence of 
the constraints on (piw) through the following optimiza- 
tion problems max \d.i + Gi(p{w)\ < Ui-aax, V« = 

1, • • • , Nm. It is straightforward now to show, using Holder's 
inequality [19, p. 29], that max \di + Gi'f{'w)\ = 

Mil + 0inax, and the result follows. ■ 

Remark 4: Problem ( [T3| l is a quadratic program in the 
optimization parameters 6 := (G, d) [20, p. Ill], and can be 
solved efficiently by standard solvers such as cvx [21]. < 

A. Examples 

An important step in the solvability of Problem ( [T3] l is 
being able to calculate the matrices Ai and A2. In general, 
these matrices can be calculated off-line by numerical inte- 
gration. However, in some instances these matrices can be 
given in terms of explicit formulas; two of these instances 
are given in the following examples. 

Recall the following standard special mathematical func- 
tions: the standard error function erf(z) :— e~^dt 
and the complementary error function [22, p. 297] defined 
by erfc(z) := 1 — erf (z) for z G M, the incomplete Gamma 
function [22, p. 260] defined by T{a, z) := t"^^e~*di for 
z,a> 0, the confluent hypergeometric function [22, p. 505] 
defined by U{a,b,z) 6-^*^-1(1 + 

for a, 6, z > and V is the standard Gamma function. 

We collect a few facts in the following 

Proposition 5: For cr^ > we have 



1) ^ 



y27r cr 
1 



2) 



e 2^2 d< 



l+t2 



\/2n a Jo 

= -(V2ncr 
2\ 



2 



erf ( 



3) -i= 



^/27^ cr 



7re 2,72 erfc 



2a 







4) -1^ 



V2'Ka 



^ erf 

00 

te 27? dt 



r(2a2,l); 



V2^aJo VT+W ^ V2' '2a2. 

Example 6: Let us consider ([Tjl with Gaussian noise and 

sigmoidal bounds on the control input. More precisely, sup- 
pose that the noise process (wf)teNo is ™ independent and 
identically distributed (i.i.d) sequence of Gaussian random 
vectors of mean and covariance S. Let the components 
of wt be mutually independent, which implies that S is a 
diagonal matrix diagjcr^, . . . , cr^}. Suppose further that the 
matrix F = I and that the function is a standard sigmoid, 
i.e., ip{t) := t/Vl + t'^ '- Then from Proposition [s] we have 
for i = 1, . . . ,n and j = 0, . . . , N — 1, 



This 
tion 

S' 



shows 

m is 



1 



v27r a. 



ire 



l + <2 
i erfc^ 

that the matrix Ai 
equal to diag{S',.-- 

diag < \/27r (Ti — 7re ^"^ 



1 



2 a, 
in 

erfc 



Proposi- 
where 



2tt an 



ne 



erfc 
2 



/2'K (Ti 



Similarly, since 
dt 



the matrix A2 in Proposition [5] is diag{E", . . . , S"}, where 
S" diag{^t/(i,0,^),...,^C/(i,0,5^)}. 
Therefore, given the system ([T]l, the control policy ( [To| , and 
the description of the noise input as above, the matrices 
Ai and A2 derived above complete the set of hypotheses 
of Proposition [3] The problem (|5]) can now be solved as a 
quadratic program ^T3\ . A 
Note that we have chosen to use the standard sigmoidal 
functions in Example [6] However, the result still holds 
for more general sigmoidal functions of the form <j){t) = 
M- 



, ' „. , where AI e M is some given magnitude and 
a € M is some given slope. This slight change is reflected in 
the entries of the matrices Ai and A2, i.e., for i = 1, . . . , n 
and j = 0, . . . , - 1, 



E[(p{w))^] = M V2^ a,a - ne 



erfc 



1 



2 (T, Q! 



and E[ip{w^j)w 

Example 7: Consider the system ([T]| as in Example [6] with 
tp being the standard saturation function defined as Lp{t) = 
sat(t) :— sgn(i) min{|t|, 1}. From Proposition |3] we have 
for i = 1 , . . . , n and j — 0, . . . , N — 1, 



2 



1 







/2tT (Ti 



2'K Ti 
^dt 



'2tt cr,- 



1 



- 2CT?e~ 



1 -l-erf 



At 



and 

2 



1 



/27r (Ti 
/2^cr?erf( 



'2tt Ui 
^dt- 



'?dt 



'2ll (7,; 



ie 



dt 



2afe 



aj:{2all). 



Therefore, in this case the matrix Ai in Proposition [3] is 
diag{S;', . . . , S'} with E' := diag{^i, . . and the ma- 

trix A2 is diag{S", . . . , E"} with E" := diagjCl', • . . 
These information complete the set of hypotheses of Proposi- 
tion |3] and the problem (|5]l can now be solved as a quadratic 
program ( [T3] l. A 

IV. Stability Analysis 

In this section, we assume that the matrix A is Schur 
stable, i.e., |Ai(A)| < 1, Vi. Accordingly, and since the 
control is bounded, it is intuitively evident that the closed- 
loop system is stable in some sense. Indeed, we shall show 
that the variance of the state is uniformly bounded both in 
the MPC and RHC cases, the only difference being a choice 
of implementation based on available memory. 

First we need the following Lemma. It is a standard variant 
of the Foster-Lyapunov condition [23]; we include a proof 
here for completeness. The hypotheses of this Lemma are 
stronger than usual, but are sufficient for our purposes; see 
e.g., [24] for more general conditions. 

Lemma 8: Let {xt)t<£No be ™ M"-valued Markov process. 
Let V : M5.0 be a continuous positive definite and 

radially unbounded function, integrable with respect to the 
probability distribution function of w. Suppose that there 
exists a compact set K C M" and a number A G ]0, 1[ such 
that 

E[V{xi)\xq^ x] i:: XV{x), yx ^ K. 

Then sup E^ [V{xt)] < 00. 

teNo 

Proof: From the conditions it follows immediately that 

E^[V{xi)] XV{x) + blK{x), VxeM" 
where b := sup E^^ [^(xi)] . We then have 



xeK 

[V{xt)] ^E^[E[V{xt)\xt^i]] 

i^ES[E[XV{xt-i) + blK{xt-i)]] 



(17) 



t-i 

i=0 



*5E, [1k{x,) 



5$ \'V{x) + 



1-A ' 



(18) 



which shows fliat sup E^, [V"(a;f )] V{x) + 6/(1 - \) < 00 

teNo 

as claimed. ■ 
We shall utilize Lemma |8] in order to show that the 
implementation of either the MPC or the RHC strategy 
generated by the solution of Problem ( [T3] l results in a 
uniformly bounded state variance. 



A. MFC Case 

The MPC implementation corresponding to our input ( [T2| ) 
and optimization program ([T3j consists of the following 
steps: Given a fixed optimization horizon N, set the initial 
time t = {), calculate the optimal control gains {G*^d*) 
using the program ( [13] ), apply the first optimal control input 



that 



"Hoc ^ ''}■ Since \mm{P) \\x\f' ^ Px, it follows 



sup 

teNo 



\Xt\ 



< 



1 



Amin(-P) tGNo 

which completes the proof. 



sup [V{xt)\ < oo, 



'o|t 



d*.^, increase t to t + 1, and iterate. Of B. RHC Case 



course, the optimal gain depends implicity on the current 
given initial state, i.e., d'^^^ = d'^^^{xt), which in turn gives 
rise to a stationary infinite horizon optimal policy given by 
^^PC (^*|„,,r*|^,...) = The closed- 

loop system is thus given by 



xt+i = Axt + Bdg^^ + Fw, 



r, 



t e 



(19) 



Proposition 9: Assume that the matrix A is Schur stable 
and the assumptions of Proposition [3] hold. Then, under 
the control policy tt'^^^ defined above, the closed loop 



\xt\ 



< oo. 



system ([19]) satisfies supfgj^|_| E^^ 

Proof: Since by assumption the matrix A is Schur 
stable, there exists a positive definite and symmetric matrix 
with real entries, say P, such that A^PA — P ^ — I„xn- 
Using the system ( [T9] ), at each time instant i G Ng we have 

E^t [xJ+iPxt+i] = 

E,, [{Axt + Bd*^^ + Fwt + rYP{Axt + Bdl^^ + Fwt + r)] 



xl A ' PAxt + 2x1 A ' P{Bdl^^ + Ffi^^ + r) 

+ d*lB^PBd*^, + r^Pr + 2(P/.„, + r^PBd^^, 

+ 2r'^PFti^^ + tr (F^PFJ:^^) . 



Using the fact that 
the following bound 



''0\t 



^ Umax (from ([13])), we obtain 



E:,^ [xJ+^Pxt+i] < xlA^PAxt + 2ci \\xt\\^ + C2, 

where ci ||ylTp(P^„^ + r)||^ + m ||ATpp||^ [/.^ax 
and C2 r^Pr + 2\\B'^ P{F ^i.^^ + r)\\^U,^,^ + 

m IIB^PPL f^max + 2kT/'i^M^J+tr(pTpFi]^J. since 
xj A^ PAxt ^ xj Pxt — xjxt, we have that 



lEj^t [a;J_|_iPxt+i] < xJPxt - \\xt\\'^ + 2ci ||a;t||^ + C2. 



(20) 



For 9 E ] max{0, 1 — Xmax{P)}, 1[ we know that 

~e \\xt\\l^^ + 2ci llxtll^ + C2 < 0, V llxtll^ > r, 



where r |(ci + ■\/c^ + C20). From ( [20] ) it now follows 
that E,J4+iPa;t+i] ^ xJ Pxt - {I - 9) \\xt\\\y \\xt\\^ > 
r, whence 



E,, [xJ+,Pxt+i] sC (l 



1 - 

-^max(-P) 



)xTpa;t, y\\xt\\^>r. 



We see that the hypotheses of Lemma [8] are satisfied with 

V{x) := a;Tpx, A := U-j^^), and := {x G 



In the RHC implementation is also iterative in nature, 
however instead of recalculating the gains at each time 
instant the optimization problem is solved every kN steps, 
where fc e No. The resulting optimal control policy (applied 
over a horizon N) is given by tt^jy — Glj^(p{Filj) + 
d^pf, where again the control gains depend implicitly on 

GlNixkN) and 



kN 



the initial condition x^n^ i-e., G 

^kN — ^kNi^kN)- Therefore, the optimal policy is given 
by TT™^ = (tt^, tt;^, • • • ). For ^ = 1, • • • , N, the resulting 
closed-loop system over horizon N is given by 

XkN+e = A''xkN + BiGlj^ip{Fw) + Bidl^ + DiFiu + Dif , 

(21) 

where k S No, and Bg and Dg are suitably defined matrices 
that are extracted from B and D, respectively. 

Proposition 10: Assume that the matrix A is Schur stable 
and the assumptions of Proposition [3] hold. Then, under 
the control policy -k^^^' defined above, the closed loop 



E, 



\xt\ 



< oo. 



system ( |2T] ) satisfies sup^gpj^ i^^o 

Proof: Using ([21) and the fact that E^. [(^(PiD)] = 0, 
Vx e M", we have that = 1, • • • ,N 

^x,^ [xl^^fPiXkN+l] = xl^{A^yPfA^XkN 
+ 2xlj^{A^y Pi{Bdlj^ + DgF^i^ + Dif) + f'^DjPtDef 
+ {dl^YBjPMlj, + 2{dlj,yBjP,D,{Ftx^ + f) 
+ 2^JilF^DlP,D,f + tv{{Gl^YBlP,B,GUK,) 



2tv {{Gl^YB]P,DiFK2) + tv{F^DjPiDtFY.^) 



Using the fact that [[d^ 



kN\ 



and G 



kN\ 



< 



C^max/0inax (from ([l3])), wc obtain the following bound 

[xkN+iBiXkN+e] 
< xlj^{A'''yPeA^-XkN + 2cu IkfewlU + ^21, 



where ci£ := 

m\\{A'yPiB,\\^U, 



\{A'yPiDi{F^i^ 



I iC/„,ax + ™ II BjP.Pf [/,Lx - 
tr{FjDjPeDeFEt,) 



[tv{GllBjP,B,Gl^K, 



and C2g f'^DjPDif 

2\\{BiyPiDi{Fpi, + f)\ 
2\f'^DjPiDiFn, I 
max 

\\Glr,\\^<U„..J4'„ _ 

2tr [Gll^ B J PiDiFA2)]. Again, since A is a Schur 
stable matrix (and hence A^) there exists a matrix 
Pe — Pj > with real valued entries that satisfies 
{A^Y PiA^ — Pi < — Inxn, and its eigenvalues are real. 
Then we have xlj^{A^)~'' PiA^XkN ^ xl^Pi-XkN-xl^XkN- 
Therefore, 

^xkN [xln+iPiXkN+t] < xlj^PeXkN - ||a;fcjv||^ 

+ 2cu\\xkN\\oo + C'2£- (22) 



For 0£ G ] max{0, 1 — Aijiax(^f)}i 1[ we know that 

-di ||a;fcAr||^ + 2cu ||a;/cJv|loo + < 0, V ||a;fcAr||^ > r^, 



where ^^(cif + \/(^J+~C2t9e) . From ( |22] l it now 

follows that Ej;^„ [xJ^^^PfXfcAT+f] < xlffPiXkN ^ (1 - 
6'^) lla^fcJvll^ ,V||a;fcAr|loo > whence 

(23) 



where A/? := ( 1 — ^^p^^ ] . Define A 



max Xi 



r' :— max r^, A :— max Amax(Pf), A := 
e=i,-- ,N-i e=i,...,N-i 

min X^in{Pe), then we can obtain using (|23| the 
conservative bound 



[^kN+ePNXkN+e] < X'xl^PNXkN, V||2:fcAr||^ > r' 



for every ^ = 1, . . . , iV - 1, where A' := a4^2^#^, and 



the A^-step bound 



' AA„,i„(Pjv) ■■ 



^XkN[^Jk+l)NPNX(k+l)N] < XNxJffPNXkN, 

V||a;fc7v|loo > "^N- (24) 

Let V/v(2:) x^ P^x. Now, following the same reasoning 
as in Lemma |8] we can establish the following bound (for 

fc e No, i,...,iv- 1) 

¥.^\VN{xkN+t)\ = ¥..j,\^[VN{xkN+t)\xkN]] 

< E,[E[X'VN{xkN) + b'lK'ixkN)]] 

< [E[X'E[VN{xkN)\xik-i)N] + b'lK'ixkw)]] 

< E^ [E[X'E[XNVN{x(^k~i)N) + WK„(x(fe„i)jv)] 

+ b'lK'{xkN)]] 

k~l 

< X'X%Vn{x) + J2 X%''''bE,[lK.{x.m)] 

i=0 



+ b'E^[lK'{xkN)] 

<A'A^VW(x) + fc^ + 6', 



1 - A 



(25) 



N 



where b := sup E^; [VAr(a;Ar)] , b' := sup E2;[VAr(x/)] for 

e = ,N -1, Kn ■■= e M"|||e||^ < tn}, and 

K' e M"| ll^ll^ < r'}. Note that the conditioning in 

the steps of p5| ) is done every N steps as the problem is not 
Markovian except then. Therefore, it follows from ( p5| l that, 

Vt kN + 1, 



sup E2,[||a;f| 
teNo 



< 



1 



\ (U N sup Ea;[VAr(xfeAr+;)] , 



which completes the proof. 



oo (26) 



C. Input-to-state Stability 

Input-to-state stability (ISS) is an interesting and important 
qualitative property of systems, dealing with input-output 
behavior. In the deterministic context [25] it generalizes the 
well-known bounded input bounded output (BIBO) property 
of linear systems [26, p. 490]. ISS provides a description 
of the behavior of a system subjected to bounded inputs. 
Here we are interested in a stochastic variant of input-to-state 
stability; see e.g., [27], [28] for other possible definitions and 
ideas (primarily in continuous-time). 

One possible way to measure the strength of stochastic in- 
puts is in terms of their covariances; sometimes their moment 
generating functions are also employed. For Gaussian noise 
it is customary to consider a suitable norm of the covariance 
matrix as a measure of its strength. The deterministic version 
of input-to-state stability deals with £oo-to-£oo gain from 
the input to the state of a system. We consider the linear 
system ([T]), and establish a natural iSS-type property from 
the control and the noise inputs to the state of the system ([T}, 
under both the MPC and the RHC strategies. 

Definition 11: The system ([TJ is input-to- state stable in 
L\ if there exist functions /3 G /C£ and a, 71, 72 S /Coo such 
that for every initial condition xq E M" and Vt G Nq we 
have 



Ej;o [adlxilD] sC f3{\\xo\\ ,i) + 7i( sup \\us 



-72(iisir) 



(27) 

where ||-|| is an appropriate matrix norm. 
One difference with the deterministic definition of ISS is 
immediately evident, namely, the presence of the function 
a inside the expectation in ( pTj i. It turns out that often it is 
more natural to arrive at an estimate of E^^^ [a(|ja;t||)] for 
some a G /Coo than an estimate of Ea;^ [||xt||]. Moreover, 
in case a is convex, Jensen's inequality [29, p. 348] shows 



that such an estimate implies an estimate of E^ 



The 



following proposition can be easily established with the aid 
of Proposition [9] and Proposition 10 



Proposition 12: The closed-loop systems ([T9| and pT| ) 
are input-to-state stable in £1. ■ 
The proof is omitted for space limitations. 

V. Numerical Example 
Let us consider the system ([T]i with some generic matrices 



A = 



0.. 
0.3 
0.09 



0.1 
0.3 
0.02 



0.01 
0.06 
0.5 



B = 



1 
2 

0.5 



F = I 



3x3. 



and 



r — O3X1. We simulate the system starting from 50 different 
initial conditions, all of which are sampled according to a 
uniform distribution over [—50,50]"^. The noise inputs are 
independent and identically sampled according to a normal 
distribution, w ^ A/'(0, 4I3X3), the noise saturation function 
is chosen as in Example |6] with 0niax = 5, and the input 
saturation bound C/max = 10. The optimization gain matrices 
are chosen to be Qi = 3X3x3 and Ri = 2Iixi, Vi, and the 
optimization horizon = 6. The optimization matrices are 
given by Ai = 3.302419x9, and A2 = 0.784619x9. We used 
the cvx solver [21] to handle the optimization problem ( [T3] i. 




5 10 15 20 25 30 35 40 



(a) MPC implementation 




(b) RHC implementation 

Fig. 1. MPC and RHC algorithms corresponding to the system in f|v] 
The plots correspond to the aforementioned algorithms each run from 50 
identical initial conditions distributed uniformly over [—50,50]. 



The results for the MPC implementation are shown in Figure 



1(a) and those for the RHC implementation are shown in 



Figure |l(b)[ for the full state evolution over a horizon of 40 
time steps. Finally, it is interesting to note that the MPC and 
RHC average performance indices over the 50 different runs 
are given by 3985 and 4327, respectively. 

VI. Conclusions 

In this paper, we provided a tractable optimization pro- 
gram that solves the stochastic Model Predictive Control 
and Rolling Horizon Control problems, while guaranteeing 
the satisfaction of hard bounds on the control input. We 
have showed that in both cases the resulting closed-loop 
process has bounded variance. We demonstrated that both 
implementations enjoy some qualitative notion of stochastic 
input-to-state stability. We provided several examples in 
which crucial matrices in our optimization program can be 
calculated off-line. Future direction for this research is aimed 
at lifting the current feedback strategy onto general vector 
spaces. 
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